Orthogonal neural encoding of targets and distractors supports multivariate cognitive control.

The complex challenges of our mental life require us to coordinate multiple forms of neural information processing. Recent behavioural studies have found that people can coordinate multiple forms of attention, but the underlying neural control process remains obscure. We hypothesized that the brain implements multivariate control by independently monitoring feature-specific difficulty and independently prioritizing feature-specific processing. During functional MRI, participants performed a parametric conflict task that separately tags target and distractor processing. Consistent with feature-specific monitoring, univariate analyses revealed spatially segregated encoding of target and distractor difficulty in the dorsal anterior cingulate cortex. Consistent with feature-specific attentional priority, our encoding geometry analysis revealed overlapping but orthogonal representations of target and distractor coherence in the intraparietal sulcus. Coherence representations were mediated by control demands and aligned with both performance and frontoparietal activity, consistent with top-down attention. Together, these findings provide evidence for the neural geometry necessary to coordinate multivariate cognitive control.

Effort Foraging Task reveals positive correlation between individual differences in the cost of cognitive and physical effort in humans.

Effort-based decisions, in which people weigh potential future rewards against effort costs required to achieve those rewards involve both cognitive and physical effort, though the mechanistic relationship between them is not yet understood. Here, we use an individual differences approach to isolate and measure the computational processes underlying effort-based decisions and test the association between cognitive and physical domains. Patch foraging is an ecologically valid reward rate maximization problem with well-developed theoretical tools. We developed the Effort Foraging Task, which embedded cognitive or physical effort into patch foraging, to quantify the cost of both cognitive and physical effort indirectly, by their effects on foraging choices. Participants chose between harvesting a depleting patch, or traveling to a new patch that was costly in time and effort. Participants' exit thresholds (reflecting the reward they expected to receive by harvesting when they chose to travel to a new patch) were sensitive to cognitive and physical effort demands, allowing us to quantify the perceived effort cost in monetary terms. The indirect sequential choice style revealed effort-seeking behavior in a minority of participants (preferring high over low effort) that has apparently been missed by many previous approaches. Individual differences in cognitive and physical effort costs were positively correlated, suggesting that these are perceived and processed in common. We used canonical correlation analysis to probe the relationship of task measures to self-reported affect and motivation, and found correlations of cognitive effort with anxiety, cognitive function, behavioral activation, and self-efficacy, but no similar correlations with physical effort.

Make or break: The influence of expected challenges and rewards on the motivation and experience associated with cognitive effort exertion.

Challenging goals can induce harder work but also greater stress, in turn potentially undermining goal achievement. We sought to examine how mental effort and subjective experiences thereof interact as a function of challenge level and the size of the incentives at stake. Participants performed a task that rewarded individual units of effort investment (correctly performed Stroop trials) but only if they met a threshold number of correct trials within a fixed time interval (challenge level). We varied this challenge level (Study 1, N = 40), and the rewards at stake (Study 2, N = 79), and measured variability in task performance and self-reported affect across task intervals. Greater challenge and higher rewards facilitated greater effort investment but also induced greater stress, while higher rewards (but less challenge) simultaneously induced greater positive affect. Current findings further our understanding of task demands and incentives on mental effort exertion and wellbeing.

Motivational context determines the impact of aversive outcomes on mental effort allocation.

It is well known that people will exert effort on a task if sufficiently motivated, but how they distribute these efforts across different strategies (e.g., efficiency vs. caution) remains uncertain. Past work has shown that people invest effort differently for potential positive outcomes (rewards) versus potential negative outcomes (penalties). However, this research failed to account for differences in the context in which negative outcomes motivate someone - either as punishment or reinforcement. It is therefore unclear whether effort profiles differ as a function of outcome valence, motivational context, or both. Using computational modeling and our novel Multi-Incentive Control Task, we show that the influence of aversive outcomes on one's effort profile is entirely determined by their motivational context. Participants (N:91) favored increased caution in response to larger penalties for incorrect responses, and favored increased efficiency in response to larger reinforcement for correct responses, whether positively or negatively incentivized.

Humans reconfigure target and distractor processing to address distinct task demands.

When faced with distraction, we can focus more on goal-relevant information (targets) or focus less on goal-conflicting information (distractors). How people use cognitive control to distribute attention across targets and distractors remains unclear. We address this question by developing a novel Parametric Attentional Control Task that can "tag" participants' sensitivity to target and distractor information. We use these precise measures of attention to develop a novel process model that can explain how participants control attention toward targets and distractors. Across three experiments, we find that participants met the demands of this task by independently controlling their processing of target and distractor information, exhibiting distinct adaptations to manipulations of incentives and conflict. Whereas incentives preferentially led to target enhancement, conflict in the previous trial preferentially led to distractor suppression. These distinct drivers of control altered sensitivity to targets and distractors early in the trial, promptly followed by reactive reconfiguration toward task-appropriate feature sensitivity. To provide a process-level account of these empirical findings, we develop a novel neural network model of evidence accumulation with attractor dynamics over feature weights that reconfigure target and distractor processing. These results provide a computational account of control reconfiguration that provides new insights into how multivariate attentional signals are optimized to achieve task goals. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Control adjustment costs limit goal flexibility: Empirical evidence and a theoretical account.

The ability to switch between goals is the cornerstone of human cognition and behavior. Cognitive control allows for rapid adjustments of cognition in accordance with new goals, but control adjustments come at a cost. This cost is traditionally studied in situations that demand changes to one's task, without necessitating other changes in the control state. Goal flexibility, however, often entails maintaining the same task while adjusting the amount and type of control being allocated to that task. For instance, different stages of a given task might require us to process information more or less efficiently (e.g., by varying levels of attention) and/or respond more or less cautiously (e.g., by varying response thresholds). Across four experiments, we show that such within-task control adjustments incur a performance cost, and that a dynamical systems model can explain the source of these costs. Participants performed a single cognitively demanding task (the color-word Stroop) under varying performance goals (e.g., to be fast or to be accurate). We modeled control allocation to include a dynamic process of adjusting from one's current control state to a target state for a given performance goal. By incorporating inertia into this adjustment process, our model predicts and our empirical findings confirm that people will under-shoot their target control state more (i.e., exhibit larger adjustment costs) when (a) goals switch rather than remain fixed over a block (Study 1); (b) target control states are more distant from one another (Study 2); (c) less time is given to adjust to the new goal (Study 3); and (d) when anticipating having to switch goals more frequently (Study 4). Our findings demonstrate that there is a cost to adjusting control to meet one's goal - even in the absence of a task change - and show that this cost can emerge directly from the dynamics of control adjustment. In so doing, they shed new light on the sources of and constraints on flexibility in human goal-directed behavior.

Mutual inclusivity improves decision-making by smoothing out choice's competitive edge.

Decision-making forms a central bottleneck to most of our tasks, one that people often experience as costly. To mitigate these costs, previous work has proposed adjusting one's threshold for deciding (e.g., satisficing) to avoid over-deliberating. Here, we test an alternative solution to these costs, one that targets the basis for most choice costs: the fact that choosing one option sacrifices others (mutual exclusivity). Across 4 studies (N = 385), we test whether this tension can be relieved by framing choices as inclusive (allowing more than one option from a set, similar to a buffet), and whether doing so improves decision-making and the experience thereof. We find that inclusivity makes choices more efficient, because of its unique impact on the level of competition between potential responses as participants accumulate information for each of their options (resulting in a more "race"-like decision process). We find that inclusivity also reduces the subjective costs associated with choice, making people feel less conflicted in conditions where it was hard to choose which good option to acquire or which bad option to get rid of. These inclusivity benefits were distinct from those achieved when trying to merely reduce deliberation (e.g., tightening one's deadline), which we show can in some cases lead to similar increases in efficiency but only carry the potential to diminish not improve the experience of choosing. This work collectively provides key mechanistic insights into the conditions under which decision making is most costly, and a novel approach aimed at mitigating those costs.

Learning how to reason and deciding when to decide.

Research on human reasoning has both popularized and struggled with the idea that intuitive and deliberate thoughts stem from two different systems, raising the question how people switch between them. Inspired by research on cognitive control and conflict monitoring, we argue that detecting the need for further thought relies on an intuitive, context-sensitive process that is learned in itself.

Learning when effort matters: neural dynamics underlying updating and adaptation to changes in performance efficacy.

To determine how much cognitive control to invest in a task, people need to consider whether exerting control matters for obtaining rewards. In particular, they need to account for the efficacy of their performance-the degree to which rewards are determined by performance or by independent factors. Yet it remains unclear how people learn about their performance efficacy in an environment. Here we combined computational modeling with measures of task performance and EEG, to provide a mechanistic account of how people (i) learn and update efficacy expectations in a changing environment and (ii) proactively adjust control allocation based on current efficacy expectations. Across 2 studies, subjects performed an incentivized cognitive control task while their performance efficacy (the likelihood that rewards are performance-contingent or random) varied over time. We show that people update their efficacy beliefs based on prediction errors-leveraging similar neural and computational substrates as those that underpin reward learning-and adjust how much control they allocate according to these beliefs. Using computational modeling, we show that these control adjustments reflect changes in information processing, rather than the speed-accuracy tradeoff. These findings demonstrate the neurocomputational mechanism through which people learn how worthwhile their cognitive control is.

Disentangling choice value and choice conflict in sequential decisions under risk.

Recent years have witnessed a surge of interest in understanding the neural and cognitive dynamics that drive sequential decision making in general and foraging behavior in particular. Due to the intrinsic properties of most sequential decision-making paradigms, however, previous research in this area has suffered from the difficulty to disentangle properties of the decision related to (a) the value of switching to a new patch versus, which increases monotonically, and (b) the conflict experienced between choosing to stay or leave, which first increases but then decreases after reaching the point of indifference between staying and switching. Here, we show how the same problems arise in studies of sequential decision-making under risk, and how they can be overcome, taking as a specific example recent research on the 'pig' dice game. In each round of the 'pig' dice game, people roll a die and accumulate rewards until they either decide to proceed to the next round or lose all rewards. By combining simulation-based dissections of the task structure with two experiments, we show how an extension of the standard paradigm, together with cognitive modeling of decision-making processes, allows to disentangle properties related to either switch value or choice conflict. Our study elucidates the cognitive mechanisms of sequential decision making and underscores the importance of avoiding potential pitfalls of paradigms that are commonly used in this research area.

Rat Anterior Cingulate Cortex Continuously Signals Decision Variables in a Patch Foraging Task.

In patch foraging tasks, animals must decide whether to remain with a depleting resource or to leave it in search of a potentially better source of reward. In such tasks, animals consistently follow the general predictions of optimal foraging theory (the marginal value theorem; MVT): to leave a patch when the reward rate in the current patch depletes to the average reward rate across patches. Prior studies implicate an important role for the anterior cingulate cortex (ACC) in foraging decisions based on MVT: within single trials, ACC activity increases immediately preceding foraging decisions, and across trials, these dynamics are modulated as the value of staying in the patch depletes to the average reward rate. Here, we test whether these activity patterns reflect dynamic encoding of decision-variables and whether these signals are directly involved in decision-making. We developed a leaky accumulator model based on the MVT that generates estimates of decision variables within and across trials, and tested model predictions against ACC activity recorded from male rats performing a patch foraging task. Model predicted changes in MVT decision variables closely matched rat ACC activity. Next, we pharmacologically inactivated ACC in male rats to test the contribution of these signals to decision-making. ACC inactivation had a profound effect on rats' foraging decisions and response times (RTs) yet rats still followed the MVT decision rule. These findings indicate that the ACC encodes foraging-related variables for reasons unrelated to patch-leaving decisions.SIGNIFICANCE STATEMENT The ability to make adaptive patch-foraging decisions, to remain with a depleting resource or search for better alternatives, is critical to animal well-being. Previous studies have found that anterior cingulate cortex (ACC) activity is modulated at different points in the foraging decision process, raising questions about whether the ACC guides ongoing decisions or serves a more general purpose of regulating cognitive control. To investigate the function of the ACC in foraging, the present study developed a dynamic model of behavior and neural activity, and tested model predictions using recordings and inactivation of ACC. Findings revealed that ACC continuously signals decision variables but that these signals are more likely used to monitor and regulate ongoing processes than to guide foraging decisions.

Humans can navigate complex graph structures acquired during latent learning.

Humans appear to represent many forms of knowledge in associative networks whose nodes are multiply connected, including sensory, spatial, and semantic. Recent work has shown that explicitly augmenting artificial agents with such graph-structured representations endows them with more human-like capabilities of compositionality and transfer learning. An open question is how humans acquire these representations. Previously, it has been shown that humans can learn to navigate graph-structured conceptual spaces on the basis of direct experience with trajectories that intentionally draw the network contours (Schapiro, Kustner, & Turk-Browne, 2012; Schapiro, Turk-Browne, Botvinick, & Norman, 2016), or through direct experience with rewards that covary with the underlying associative distance (Wu, Schulz, Speekenbrink, Nelson, & Meder, 2018). Here, we provide initial evidence that this capability is more general, extending to learning to reason about shortest-path distances across a graph structure acquired across disjoint experiences with randomized edges of the graph - a form of latent learning. In other words, we show that humans can infer graph structures, assembling them from disordered experiences. We further show that the degree to which individuals learn to reason correctly and with reference to the structure of the graph corresponds to their propensity, in a separate task, to use model-based reinforcement learning to achieve rewards. This connection suggests that the correct acquisition of graph-structured relationships is a central ability underlying forward planning and reasoning, and may be a core computation across the many domains in which graph-based reasoning is advantageous.

Cognitive Control as a Multivariate Optimization Problem.

A hallmark of adaptation in humans and other animals is our ability to control how we think and behave across different settings. Research has characterized the various forms cognitive control can take-including enhancement of goal-relevant information, suppression of goal-irrelevant information, and overall inhibition of potential responses-and has identified computations and neural circuits that underpin this multitude of control types. Studies have also identified a wide range of situations that elicit adjustments in control allocation (e.g., those eliciting signals indicating an error or increased processing conflict), but the rules governing when a given situation will give rise to a given control adjustment remain poorly understood. Significant progress has recently been made on this front by casting the allocation of control as a decision-making problem. This approach has developed unifying and normative models that prescribe when and how a change in incentives and task demands will result in changes in a given form of control. Despite their successes, these models, and the experiments that have been developed to test them, have yet to face their greatest challenge: deciding how to select among the multiplicity of configurations that control can take at any given time. Here, we will lay out the complexities of the inverse problem inherent to cognitive control allocation, and their close parallels to inverse problems within motor control (e.g., choosing between redundant limb movements). We discuss existing solutions to motor control's inverse problems drawn from optimal control theory, which have proposed that effort costs act to regularize actions and transform motor planning into a well-posed problem. These same principles may help shed light on how our brains optimize over complex control configuration, while providing a new normative perspective on the origins of mental effort.

Aversive motivation and cognitive control.

Aversive motivation plays a prominent role in driving individuals to exert cognitive control. However, the complexity of behavioral responses attributed to aversive incentives creates significant challenges for developing a clear understanding of the neural mechanisms of this motivation-control interaction. We review the animal learning, systems neuroscience, and computational literatures to highlight the importance of experimental paradigms that incorporate both motivational context manipulations and mixed motivational components (e.g., bundling of appetitive and aversive incentives). Specifically, we postulate that to understand aversive incentive effects on cognitive control allocation, a critical contextual factor is whether such incentives are associated with negative reinforcement or punishment. We further illustrate how the inclusion of mixed motivational components in experimental paradigms enables increased precision in the measurement of aversive influences on cognitive control. A sharpened experimental and theoretical focus regarding the manipulation and assessment of distinct motivational dimensions promises to advance understanding of the neural, monoaminergic, and computational mechanisms that underlie the interaction of motivation and cognitive control.

Advances in modeling learning and decision-making in neuroscience.

An organism's survival depends on its ability to learn about its environment and to make adaptive decisions in the service of achieving the best possible outcomes in that environment. To study the neural circuits that support these functions, researchers have increasingly relied on models that formalize the computations required to carry them out. Here, we review the recent history of computational modeling of learning and decision-making, and how these models have been used to advance understanding of prefrontal cortex function. We discuss how such models have advanced from their origins in basic algorithms of updating and action selection to increasingly account for complexities in the cognitive processes required for learning and decision-making, and the representations over which they operate. We further discuss how a deeper understanding of the real-world complexities in these computations has shed light on the fundamental constraints on optimal behavior, and on the complex interactions between corticostriatal pathways to determine such behavior. The continuing and rapid development of these models holds great promise for understanding the mechanisms by which animals adapt to their environments, and what leads to maladaptive forms of learning and decision-making within clinical populations.

Dissociable influences of reward and punishment on adaptive cognitive control.

To invest effort into any cognitive task, people must be sufficiently motivated. Whereas prior research has focused primarily on how the cognitive control required to complete these tasks is motivated by the potential rewards for success, it is also known that control investment can be equally motivated by the potential negative consequence for failure. Previous theoretical and experimental work has yet to examine how positive and negative incentives differentially influence the manner and intensity with which people allocate control. Here, we develop and test a normative model of control allocation under conditions of varying positive and negative performance incentives. Our model predicts, and our empirical findings confirm, that rewards for success and punishment for failure should differentially influence adjustments to the evidence accumulation rate versus response threshold, respectively. This dissociation further enabled us to infer how motivated a given person was by the consequences of success versus failure.

Decomposing the motivation to exert mental effort.

Achieving most goals demands cognitive control, yet people vary widely in their success at meeting these demands. While motivation is known to be fundamental to determining these successes, what determines one's motivation to perform a given task remains poorly understood. Here, we describe recent efforts towards addressing this question using the Expected Value of Control model, which simulates the process by which people weigh the costs and benefits of exerting mental effort. By functionally decomposing this cost-benefit analysis, this model has been used to fill gaps in our understanding of the mechanisms of mental effort and to generate novel predictions about the sources of variability in real-world performance. We discuss the opportunities the model provides for formalizing hypotheses about why people vary in their motivation to perform tasks, as well as for understanding limitations in our ability to test these hypotheses based on a given measure of performance.

Learning to Overexert Cognitive Control in a Stroop Task.

How do people learn when to allocate how much cognitive control to which task? According to the Learned Value of Control (LVOC) model, people learn to predict the value of alternative control allocations from features of a situation. This suggests that people may generalize the value of control learned in one situation to others with shared features, even when demands for control are different. This makes the intriguing prediction that what a person learned in one setting could cause them to misestimate the need for, and potentially overexert, control in another setting, even if this harms their performance. To test this prediction, we had participants perform a novel variant of the Stroop task in which, on each trial, they could choose to either name the color (more control-demanding) or read the word (more automatic). Only one of these tasks was rewarded each trial and could be predicted by one or more stimulus features (the color and/or word). Participants first learned colors and then words that predicted the rewarded task. Then, we tested how these learned feature associations transferred to novel stimuli with some overlapping features. The stimulus-task-reward associations were designed so that for certain combinations of stimuli, transfer of learned feature associations would incorrectly predict that more highly rewarded task would be color-naming, even though the actually rewarded task was word-reading and therefore did not require engaging control. Our results demonstrated that participants overexerted control for these stimuli, providing support for the feature-based learning mechanism described by the LVOC model.

Temporal discounting correlates with directed exploration but not with random exploration.

The explore-exploit dilemma describes the trade off that occurs any time we must choose between exploring unknown options and exploiting options we know well. Implicit in this trade off is how we value future rewards - exploiting is usually better in the short term, but in the longer term the benefits of exploration can be huge. Thus, in theory there should be a tight connection between how much people value future rewards, i.e. how much they discount future rewards relative to immediate rewards, and how likely they are to explore, with less 'temporal discounting' associated with more exploration. By measuring individual differences in temporal discounting and correlating them with explore-exploit behavior, we tested whether this theoretical prediction holds in practice. We used the 27-item Delay-Discounting Questionnaire to estimate temporal discounting and the Horizon Task to quantify two strategies of explore-exploit behavior: directed exploration, where information drives exploration by choice, and random exploration, where behavioral variability drives exploration by chance. We find a clear correlation between temporal discounting and directed exploration, with more temporal discounting leading to less directed exploration. Conversely, we find no relationship between temporal discounting and random exploration. Unexpectedly, we find that the relationship with directed exploration appears to be driven by a correlation between temporal discounting and uncertainty seeking at short time horizons, rather than information seeking at long horizons. Taken together our results suggest a nuanced relationship between temporal discounting and explore-exploit behavior that may be mediated by multiple factors.

A computational perspective on the roles of affect in cognitive control.

Previous work has demonstrated that cognitive control can be influenced by affect, both when it is tied to the anticipated outcomes for cognitive performance (integral affect) and when affect is induced independently of performance (incidental affect). However, the mechanisms through which such interactions occur remain debated, in part because they have yet to be formalized in a way that allows experimenters to test quantitative predictions of a putative mechanism. To generate such predictions, we leveraged a recent model that determines cognitive control allocation by weighing potential costs and benefits in order to determine the overall Expected Value of Control (EVC). We simulated potential accounts of how integral and incidental affect might influence this valuation process, including whether incidental positive affect influences how difficult one perceives a task to be, how effortful it feels to exert control, and/or the marginal utility of succeeding at the task. We find that each of these accounts makes dissociable predictions regarding affect's influence on control allocation and measures of task performance (e.g., conflict adaptation, switch costs). We discuss these findings in light of the existing empirical findings and theoretical models. Collectively, this work grounds existing theories regarding affect-control interactions, and provides a method by which specific predictions of such accounts can be confirmed or refuted based on empirical data.